1. Technical Field
The present invention relates to an image pickup device, an integrated circuit of an image pickup element, and an image pickup result processing method, and is applicable to for example video cameras recording an image pickup result by moving picture, electronic still cameras, monitoring devices and the like. The present invention connects and thereby integrates image pickup means and image compressing means with each other by a wiring layer formed on a surface on an opposite side from an image pickup surface of image pickup means, changes a data compression ratio used for data compression of at least a next block on the basis of an amount of code generated by data compression of image pickup results in a block unit, and outputs an image pickup result of each photoelectric conversion unit from the image pickup means in such a manner as to correspond to the processing in a block unit, whereby it is possible to make effective use of a high degree of freedom of reading image pickup results, which is a feature of a CMOS solid-state image pickup element or the like, simplify a general configuration, and reliably perform rate control.
2. Background Art
A video camera in related art buffers an image pickup result output from a CCD (Charge Coupled Device) solid-state image pickup element in a frame memory, and data-compresses the image pickup result in block units by an MPEG (Moving Picture Experts Group) method. In this MPEG2, rate control is performed by for example a method of TM5 (Test Mode 5) or the like. Similarly, an electronic still camera buffers an image pickup result output from a CCD solid-state image pickup element in a frame memory, data-compresses the image pickup result in block units by a JPEG (Joint Photographic Coding Experts Group) method, and performs rate control as in MPEG2.
On the other hand, CMOS solid-state image pickup elements have recently been put to practical use. As shown in FIG. 1 by contrast with the CCD solid-state image pickup element, the CMOS solid-state image pickup element has various features. For example, with the CCD solid-state image pickup element, times of a start and an end of charge accumulation are the same in all pixels, whereas with the CMOS solid-state image pickup element, times of a start and an end of charge accumulation are different in column or pixel units.
In particular, as shown in FIG. 2, the CCD solid-state image pickup element reads image pickup results of respective pixels by serial transmission. On the other hand, as shown in FIG. 3, the CMOS solid-state image pickup element can read image pickup results of respective pixels by X-Y address control, and thus has a high degree of freedom of reading image pickup results as compared with the CCD solid-state image pickup element. FIG. 2 is a schematic diagram showing output of image pickup results from the CCD solid-state image pickup element. An accumulated charge retained in each pixel is transferred to a vertical transfer register. The CCD solid-state image pickup element sequentially transfers the accumulated charge transferred to the vertical transfer register to a horizontal transfer register, and sequentially transfers and outputs the accumulated charge by the horizontal transfer register. On the other hand, FIG. 3 is a schematic diagram of output of image pickup results from the CMOS solid-state image pickup element, representing a case where image pickup results of respective pixels are sequentially output in column line units. In this case, the image pickup results can be output in a simultaneous and parallel manner in units equal in number to column lines.
Specifically, the CMOS solid-state image pickup element selectively turns on a MOSFET provided in each pixel by a horizontal address line extending in a horizontal direction and a vertical address line extending in a vertical direction, whereby an image pickup result is output to a signal line from the pixel selected by the horizontal address line and the vertical address line. Thus, in the example shown in FIG. 3, a plurality of pixels contiguous in the vertical direction share a signal line formed by one column line. By sequentially changing settings of horizontal address lines for the plurality of pixels connected to one column line and thereby sequentially turning on MOSFETs provided in the plurality of pixels, the column line is allocated to the pixels contiguous in the vertical direction by time division, and image pickup results of these pixels are output. As viewed in the horizontal direction, pixels contiguous in the horizontal direction share a horizontal address line. Such time-division allocation of pixels contiguous in the vertical direction to a column line is performed in pixels contiguous in the horizontal direction in a simultaneous and parallel manner, whereby image pickup results are output in line units.
A constitution in which such a CMOS solid-state image pickup element is integrated with a peripheral circuit is proposed in Japanese Patent Laid-open No. 2004-31785, for example.
On the other hand, various coding methods using a wavelet transform process have recently been proposed as coding systems for processing such image data. In the wavelet transform process, the image data is band-divided and down-sampled into a high-frequency component and a low-frequency component in each of a horizontal direction and a vertical direction, whereby the image data is divided into four subbands. There are for example a case where as shown in FIG. 4(A), the image data is processed by four subbands HH, HL, LH, and LL by performing such a dividing process only once, and a case where as shown in FIG. 4(B), the image data is processed by repeating the band dividing process. Incidentally, FIG. 4(B) represents a case where the band dividing process is repeated three times. In this example, of the subbands HH, HL, LH, and LL, the subband LL of low frequencies in the horizontal direction and the vertical direction is further subjected to the band dividing process to generate four subbands LLHH, LLHL, LLLH, and LLLL. Of the subbands LLHH, LLHL, LLLH, and LLLL, the subband LLLL of low frequencies in the horizontal direction and the vertical direction is further subjected to the band dividing process to generate four subbands LLLLHH, LLLLHL, LLLLLH, and LLLLLL.
Proposed as a coding process based on such a wavelet transform process are a so-called line-based wavelet transform in which image data is processed in line units and a so-called tile-based wavelet transform in which image data is processed in tile units as rectangular blocks set by dividing one screen.
It is considered that when the high degree of freedom of reading image pickup results, which is a feature of a CMOS solid-state image pickup element, can be utilized effectively, it is possible to further simplify the general configuration of the image pickup device, and perform rate control reliably.